Image search apparatus, method of controlling operation of same, and image search server

ABSTRACT

An image search apparatus includes a display control device, a feature quantity calculation device, a scoring device for scoring the image based upon the values of the feature quantities calculated by the feature quantity calculation device, a first scoring control device, responsive to application of a first move command which moves an image being displayed in the candidate area to a search result area, for controlling the scoring device to raise the value of feature quantities, which correspond to the feature quantities of the image for which the first move command has been applied, and score the multiplicity of images based upon the raised values of the feature quantities, and an image placement decision device for deciding image placement in such a manner that a predetermined number of images having high scores obtained are displayed in the search result area, and other images are displayed in the candidate area.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of PCT International Application No.PCT JP2014/055961 filed on Mar. 7, 2014, which claims priority under 35U.S.C. §119(a) to Japanese Patent Application No. 2013-068199 filed Mar.28, 2013. Each of the above application(s) is hereby expresslyincorporated by reference, in its entirety, into the presentapplication.

BACKGROUND OF THE INVENTION

Field of the Invention

This invention relates to an image search apparatus, a method ofcontrolling the operation thereof, and an image search server.

Description of the Related Art

Japanese Patent Application Laid-Open No. 2011-103135 describes a systemin which, when an electronic album is created, images to be placed in alayout frame provided in a template are selected automatically and theselected images are then placed in the layout frame.

According to the description set forth in Japanese Patent ApplicationLaid-Open No. 2011-103135, the system is such that when an automaticallycreated album is revised, desired images are re-selected from among alarge number of images. This makes it difficult for the user to findimages in line with user preference.

SUMMARY OF THE INVENTION

The present invention provides an image search apparatus, an imagesearch server, and a method of controlling operation of an image searchapparatus in which an image in line with user preference is foundcomparatively simply.

An image search apparatus according to a first aspect of the presentinvention includes: a display control device (display control means) forcontrolling a display unit so as to display a plurality of images in acandidate area of a display screen; a feature quantity calculationdevice (feature quantity calculation means) for calculating, with regardto each image of a multiplicity of images, the values of featurequantities representing characteristics of the image; a scoring device(scoring means) for scoring the image based upon the values of thefeature quantities calculated by the feature quantity calculationdevice; a first scoring control device (first scoring control means),responsive to application of a first move command which moves an imagebeing displayed in the candidate area to a search result area, forcontrolling the scoring device so as to raise the value of featurequantities, which correspond to the feature quantities of the image forwhich the first move command has been applied, and score themultiplicity of images based upon the raised values of the featurequantities; and an image placement decision device (image placementdecision means) for deciding image placement in such a manner that apredetermined number of images having high scores obtained by thescoring device are displayed in the search result area of the displayscreen, and images other than the predetermined number of images aredisplayed in the candidate area of the display screen.

The first aspect of the present invention provides also an operationcontrol method suited to an image search apparatus. Specifically, themethod includes steps of: controlling a display unit so as to display aplurality of images in a candidate area of a display screen;calculating, with regard to each image of a multiplicity of images, thevalues of feature quantities representing characteristics of the image;scoring the image based upon the calculated values of the featurequantities; in response to application of a first move command whichmoves an image being displayed in the candidate area to a search resultarea, raising the value of feature quantities, which correspond to thefeature quantities of the image for which the first move command hasbeen applied, and scoring the multiplicity of images based upon theraised values of the feature quantities; and deciding image placement insuch a manner that a predetermined number of images having high scoresobtained by scoring are displayed in the search result area of thedisplay screen, and images other than the predetermined number of imagesare displayed in the candidate area of the display screen. It may bearranged so as to create an electronic album using images that are beingdisplayed in the search result area.

The first aspect of the present invention provides also a recordingmedium storing a program for controlling a computer of an image searchapparatus.

In accordance with the present invention, a plurality of images aredisplayed in a candidate area of a display screen. With regard to eachimage of a multiplicity of images, the values of feature quantitiesrepresenting the characteristics of the image are calculated and theimage is scored based upon the calculated values of the featurequantities. If an image being displayed in the candidate area is movedto a search result area, the value of feature quantities, whichcorrespond to the feature quantities of the image for which the firstmove command has been applied, are raised, and the images are scoredbased upon the raised values of the feature quantities. Image placementis decided in such a manner that a predetermined number of images havinghigh scores are displayed in the search result area of the displayscreen while images other than the predetermined number of images aredisplayed in the candidate area of the display screen. An image that ismoved to the search result area is construed to be an image in line withuser preference, and images having feature quantities the same as thoseof this image are displayed in the search result area. As a result,images in line with user preference are displayed in the search resultarea.

By way of example, the display control device, in response toapplication of an image display command to display images in theplacement decided by the image placement decision device, controls thedisplay unit so as to display images in the decided placement.

The display control device may control the display unit so as to displaya deletion area, in addition to the candidate area, on the displayscreen. In this case, it is preferred that the apparatus furtherincludes a second scoring control device (second scoring control means),responsive to application of a second move command which moves an imagebeing displayed in the candidate area or an image being displayed in thesearch result area to the deletion area, for controlling the scoringdevice so as to lower the value of feature quantities, which correspondto the feature quantities of the image for which the second move commandhas been applied, and score the multiplicity of images based upon thelowered values of feature quantities.

By way of example, the first scoring control device, in response toapplication of a third move command which moves an image, which is beingdisplayed in the search result area in accordance with the first movecommand, to the candidate area, controls the scoring device so as toscore the images upon restoring raised values of the feature quantitiesto their original values. By way of example, the second scoring controldevice, in response to application of a third move command which movesan image, which is being displayed in the search result area inaccordance with the second move command, to the candidate area, controlsthe scoring device so as to score the images upon restoring loweredvalues of the feature quantities to their original values.

The apparatus may further includes an electronic album creation device(electronic album creation means) for creating an electronic album usingimages being displayed in the search result area by the image placementdecision device.

A second aspect of the present invention provides an image search serverconstituting an image search system having a client computer and theimage search server, including: a feature quantity calculation device(feature quantity calculation means) for calculating, with regard toeach of a multiplicity of images, the values of feature quantitiesrepresenting characteristics of the image; a scoring device (scoringmeans) for scoring the image based upon the values of the featurequantities calculated by the feature quantity calculation device; and afirst scoring control device (first scoring control means), responsiveto application of a first move command which moves one image among aplurality of images being displayed in a candidate area, which has beenformed on a display screen of the client computer, to a search resultarea, for controlling the scoring device so as to raise the value offeature quantities, which correspond to the feature quantities of theone image for which the first move command has been applied, and scorethe multiplicity of images based upon the raised values of the featurequantities.

The second aspect of the present invention provides also an operationcontrol method suited to an image search server. Specifically, thesecond aspect of the present invention provides a method of controllingoperation of an image search server constituting an image search systemhaving a client computer and the image search server, including stepsof: calculating, with regard to each image of a multiplicity of images,the values of feature quantities representing characteristics of theimage; scoring the image based upon the calculated values of the featurequantities; and in response to application of a first move command whichmoves one image among a plurality of images being displayed in acandidate area, which has been formed on a display screen of the clientcomputer, to a search result area, raising the value of featurequantities, which correspond to the feature quantities of the one imagefor which the first move command has been applied, and scoring themultiplicity of images based upon the raised values of the featurequantities.

In the second aspect of the present invention, images in line with userpreference are displayed in the search result area in a manner similarto that of the first aspect of the present invention.

The server may further includes a second scoring control device (secondscoring control means), responsive to application of a second movecommand which moves one image among a plurality of images beingdisplayed in the candidate area, which has been formed on the displayscreen of the client computer, or one image among a plurality of imagesbeing displayed in the search result area, to a deletion area, forcontrolling the scoring device so as to lower the value of featurequantities, which correspond to the feature quantities of the image forwhich the second move command has been applied, and score themultiplicity of images based upon the lowered values of featurequantities.

A third aspect of the present invention provides an image search serverconstituting an image search system having a client computer and theimage search server, including: a feature quantity calculation device(feature quantity calculation means) for calculating, with regard toeach image of a multiplicity of images, the values of a featurequantities representing characteristics of the image; a scoring device(scoring means) for scoring the image based upon the values of thefeature quantities calculated by the feature quantity calculationdevice; wherein a plurality of images having high scores obtained by thescoring device are displayed in a search result area of a display screenof the client computer and images other than the plurality of images aredisplayed in a candidate area of the display screen; and a first scoringcontrol device (first scoring control means), responsive to applicationof a first move command which moves an image being displayed in thecandidate area, which has been formed on the display screen of theclient computer, to the search result area, for controlling the scoringdevice so as to raise the value of feature quantities, which correspondto the feature quantities of the image for which the first move commandhas been applied, and score the multiplicity of images based upon theraised values of the feature quantities.

The third aspect of the present invention provides also an operationcontrol method suited to an image search server. Specifically, the thirdaspect of the present invention provides a method of controllingoperation of an image search server constituting an image search systemhaving a client computer and the image search server, including stepsof: calculating, with regard to each image of a multiplicity of images,the values of feature quantities representing characteristics of theimage; scoring the image based upon the calculated values of the featurequantities; wherein a plurality of images having high scores obtained byscoring are displayed in a search result area of a display screen of theclient computer and images other than the plurality of images aredisplayed in a candidate area of the display screen; and in response toapplication of a first move command which moves an image being displayedin the candidate area, which has been formed on the display screen ofthe client computer, to the search result area, raising the value offeature quantities, which correspond to the feature quantities of theimage for which the first move command has been applied, and scoring themultiplicity of images based upon the raised values of the featurequantities.

In the third aspect of the present invention as well, images in linewith user preference are displayed in the search result area in a mannersimilar to that of the first aspect of the present invention.

The server may further includes a second scoring control device (secondscoring control means), responsive to application of a second movecommand which moves an image being displayed in the candidate area,which has been formed on the display screen of the client computer, oran image being displayed in the search result area, to a deletion area,for controlling the scoring device so as to lower the value of featurequantities, which correspond to the feature quantities of the image forwhich the second move command has been applied, and score themultiplicity of images based upon the raised values of the featurequantities.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating the configuration of a photobookcreation apparatus;

FIGS. 2 and 3 are flowcharts illustrating processing executed by thephotobook creation apparatus;

FIG. 4 illustrates how images are grouped;

FIG. 5 is an example of image lists;

FIG. 6 is an example of a table of image feature quantities;

FIG. 7 is an example of a table of weighting coefficients of featurequantities;

FIG. 8 is an example of a list of unviewed pages;

FIG. 9 illustrates image scoring;

FIG. 10 is an example of a virtual editing image;

FIG. 11 is a flowchart of page editing processing;

FIG. 12 is an example of an editing image;

FIG. 13 illustrates the configuration of a photobook creation system;

FIGS. 14 to 17 are flowcharts illustrating processing executed by thephotobook creation system;

FIG. 18 is a flowchart of page editing processing;

FIGS. 19 to 22 are flowcharts illustrating processing executed by aclient computer and image search server; and

FIG. 23 is an example of a search result image.

DESCRIPTION OF THE EMBODIMENTS

FIG. 1, which shows an embodiment of the present invention, is a blockdiagram illustrating the electrical configuration of a photobook(electronic album) creation 1.

The overall operation of the photobook creation apparatus 1 iscontrolled by a CPU 2.

The photobook creation apparatus 1 includes a CD-ROM(Compact-Disc-Read-Only Memory) drive 6. By loading a CD-ROM 7containing an operation program, described later, in the CD-ROM drive 6,the operation program is read and installed in the photobook creationapparatus 1. The photobook creation apparatus 1 is provided with acommunication unit 10 as well. The operation program may be received bythe communication unit 10 and the received operation program installedin the photobook creation apparatus 1.

The photobook creation apparatus 1 includes a display device 4controlled by a display control unit 3. A touch panel 5 has been formedon the display screen of the display device 4. A signal produced inresponse to touching of the touch panel 5 is input to the CPU 2. Thephotobook creation apparatus 1 is further provided with a hard disk 12,which is accessed by a hard-disk drive 11, and a memory card reader 8.When a memory card 9 storing image files representing a large number ofimages is loaded in the memory card reader 8, the image files are readby the memory card reader 8.

FIGS. 2 and 3 are flowcharts illustrating processing executed by thephotobook creation apparatus 1.

The photobook creation apparatus 1 is so adapted that in a case wherethe photobook creation apparatus 1 according to this embodiment displaysthe automatically laid-out page of a photobook on a display screen and auser performs an editing operation for changing an image, the photobookcreation apparatus 1 refers to the editing operation and arranges it sothat images to be laid out on the page of the photobook will be imagesin line with user preference.

The period of image capture of images to be included in the photobook isdesignated by the user (step 21). The memory card 9 (which may be someother medium) is loaded in the memory card reader 8 and image filesrepresenting a large number of images stored on the memory card 9 areread (step 22). The large number of images represented by the read imagefiles are grouped based upon the date and time of image capture and animage list is formed for every group (step 23).

FIG. 4 illustrates how a large number of read images are grouped.

The horizontal axis in FIG. 4 indicates date and time of image capture.A large number of read images i are arrayed based upon date and time ofimage capture and are grouped according to image-capture date and timesthat are close together. In FIG. 4, 11 groups, namely group G1 to groupG11, have been created. Although grouping is performed in such a mannerthat the numbers of images contained in the groups are approximately thesame, they need not necessarily be the same. It goes without saying thatthe date and time of image capture will have been recorded in the headerof each image file.

When the images are grouped, an image list is created for every group.

FIG. 5 is an example of image lists of the groups G1 and G2.

In group G1, 100 images from images I1 to I100 form this one group andan image list of these images is created. In group G2, 113 images fromimages I101 to I213 form this one group and an image list of theseimages is created. Image lists are created in similar fashion for theother groups as well.

With reference again to FIG. 2, values of feature quantities with regardto all images of all image lists are calculated (step 24).

FIG. 6 is an example of a table of image feature quantities regardinggroup G2.

The image feature quantity table contains values of feature quantitiesof the images, which are included in the image list, in correspondencewith feature quantities (blurriness, brightness, composition, etc.)representing the characteristics of the images. For example, the valuesof the feature quantities blurriness, brightness and composition of theimage I101 are u101, b101 and a101, respectively. Similarly, the valuesof the feature quantities blurriness, brightness and composition of theimage I102 are u102, b102 and a102, respectively. The table similarlycontains values of feature quantities regarding the other images I103 toI212 included in group G1.

By calculating the values of the feature quantities of all images in allof the image lists, image feature quantity tables of the above kind arecreated with regard to all of the groups G1 to G21 obtained by groupingof the images.

Next, a photobook page number n is set (step 25 in FIG. 2). Thephotobook page number n indicates a page that will be edited by theuser, and the page specified by the page number n is displayed on thedisplay screen (see FIG. 12). Further, feature quantity weightingcoefficients are initialized (step 26).

FIG. 7 is an example of a table of feature quantity weightingcoefficients.

The feature quantity weighting coefficient table contains weightingcoefficients in correspondence with feature quantities. The value of afeature quantity (see FIG. 5) is multiplied by the weightingcoefficient. As will be described later, the sum total of values offeature quantities of an image multiplied by weighting coefficients isthe score of the image.

As will be described later in greater detail, weighting coefficients arechanged in accordance with the editing operation performed by the user.For this reason, a history of weighting coefficients that prevail afterthe performance of editing operations is stored in the feature quantityweighting coefficient table in addition to the initial values of theweighting coefficients. For example, although k is the initial value foreach of the feature quantities blurriness, brightness and composition,the weighting coefficients of the feature quantities blurriness,brightness and composition take on the values k1, k2 and k3,respectively, owing to a first-time editing operation.

Next, all pages are added to an unviewed page list (step 27 in FIG. 2).

FIG. 8 is an example of a list of unviewed pages.

The unviewed page list indicates pages of the photobook that have notyet undergone editing by the user. Since even one page will not havebeen edited by the user in the initial state, all pages of the photobookare entered in the unviewed page list. Further, in this embodiment, atwo-page spread of the photobook undergoes editing (although editingneed not necessarily be applied to a two-page spread). Accordingly,pairs of pages, excluding the first page, have been added to theunviewed page list.

Next, an nth page is deleted from the unviewed page list (step 28 inFIG. 3). Since n−1 has been set, the first page is deleted from theunviewed page list.

Next, the images in all of the image lists are scored utilizing thefeature quantity weighting coefficients (step 29 in FIG. 3).

FIG. 9 is an example of a score table indicating the scores of imagesbelonging to the image list of group G2.

The score table contains the scores in correspondence with the images.The score of an image is the sum total of the values of the imagefeature quantities obtained by multiplying the feature quantities of theimage, which are contained in the image feature quantity table of FIG.6, by the weighting coefficients of these feature quantities containedin the feature quantity weighting coefficient table of FIG. 7. Forexample, the value of the blurriness feature quantity, value of thebrightness feature quantity and value of the composition featurequantity of image I101 are u101, b101 and a101, respectively, as shownin FIG. 6, and the weighting coefficient of the blurriness featurequantity, weighting coefficient of the brightness feature quantity andweighting coefficient of the composition feature quantity are all k.Therefore, the score P101 of image I101 is calculated asP101=u101×k+b101×k+a101×k Such scoring is carried out for every imagelist (every group) with regard to the images belonging to the imagelists of all of the groups G1 to G21. Score tables are created withregard to all groups.

In this embodiment, first the photobook creation apparatus 1 performslayout with regard to pages other than page n (page 1 in this casebecause n=1 has been set) to be edited by the user.

FIG. 10 illustrates a virtual editing image 40 of page 2 and page 3included in an unviewed page list.

The virtual editing image 40 corresponds to an image actually displayedon the display screen of the display device 4 of the photobook creationapparatus 1. The virtual editing image 40 is created within thephotobook creation apparatus 1 before it is actually displayed on thedisplay screen. When an edit command regarding the created virtualediting image 40 is applied, the virtual editing image 40 is displayedon the display screen as an image for editing purposes and the userperforms a layout change or the like while viewing the displayed image.

The virtual editing image 40 includes a virtual candidate area 41, avirtual layout area 42 (a search result area of the present invention)and a virtual deletion area 43. The virtual layout area 42 is fordisplaying the layout of pages that constitute the photobook and itincludes virtual pages 44 of a two-page spread corresponding to pages ofthe photobook. Images have been assigned to a left-side virtual page(page 2) 45 and to a right-side virtual page (page 3) 46. These imagesare those having the top ten high scores from among the images containedin the image list of group G2 corresponding to page 2 and page 3, asmentioned above. Images whose scores rank 11^(th) and lower from amongthe images contained in the image list of group G2 corresponding topages 2 and 3 have been assigned to the virtual candidate area 41. Thevirtual deletion area 43 need not necessarily be utilized.

The image assigning processing shown in FIG. 10 is executed with regardto all pages contained in the unviewed page list.

With reference again to FIG. 3, images having the top ten high scores inthe two-page spread of the unviewed page list from among the imagesbelonging to the image list that corresponds to this two-page spread areassigned to (laid out in) the virtual layout area 42 (step 30), andimages whose scores rank 11^(th) and lower are assigned to the virtualcandidate area 41 (step 31).

If the editing of page n has not been completed (“NO” at step 32), pageediting is performed by the user (step 33). If editing of page n hasbeen completed (“YES” at step 32), then a check is made to determinewhether the editing of all pages of the photobook has been completed(step 34). If such editing has not been completed (“NO” at step 34), thepage number n to be edited next is designated by the user (step 35).Owing to the two-page spread, an even number would be designated. Thepage number n is set to the designated page number n (step 36) and theprocessing of the above-described steps 28 to 33 is repeated with regardto the set page number n.

FIG. 11 is a flowchart illustrating page editing processing (theprocessing executed at step 33 in FIG. 3). FIG. 12 is an example of anediting image 40A actually displayed on the display screen of thedisplay device 4 of the photobook creation apparatus 1.

The editing image of the group corresponding to an nth page is displayedon the display screen (step 51). The editing image 40A, whichcorresponds to the above-described virtual editing image 40, includes acandidate area 41A, a layout area 42A and a deletion area 43A. Thelayout area 42A includes an image 44A of a two-page spread. Imageshaving the top ten high scores are displayed in the two-page spreadimage 44A. However, the left-side page 45A of the two-page spread image44A that includes the first page of the photobook does not include anyimages, and the right-side page 46A of the two-page spread image 44Abecomes the first page of the photobook. Therefore, from among theimages included in the image list of the group of the nth page, thosehaving the top five high scores are displayed on the right-side page46A, and those whose scores rank sixth and lower are displayed in thecandidate area 41A. Images that are not required to be placed in theelectronic album are displayed in the deletion area 43A.

Using the editing image displayed on the display screen, the userchanges the images and layout. For example, if an image being displayedin the candidate area 41A or deletion area 43A is moved onto thetwo-page spread image 44A in the layout area 42A by drag-and-drop (“YES”at step 52), as indicated by arrow 47A in FIG. 12, the characteristicsof the moved image are construed to be in line with the preference ofthe user. When the editing operation is performed, therefore, theweighting coefficients of the feature quantities that correspond to thefeature quantities of the moved image are changed (step 53). Forexample, if the blurriness feature quantity among the feature quantitiesof the moved image is higher than the average feature quantity, it isunderstood that blurry images rather than highly sharp images are inline with user preference. The blurriness weighting coefficient,therefore, is raised. For example, the initial value k for theblurriness feature quantity is changed to k1. The weighting coefficientthus changed is stored in the feature quantity weighting coefficienttable. A similar operation is performed with regard to the weightingcoefficients of the other feature quantities such as brightness andcomposition.

Further, if an image being displayed in the candidate area 41A or layoutarea 42A is moved to the deletion area 43A (“YES” at step 54), asindicated by arrow 47B in FIG. 12, the characteristics of the movedimage are construed not to be in line with the preference of the user.When such an editing operation is performed, the weighting coefficientsof the feature quantities that correspond to the feature quantities ofthe moved image are lowered (step 55).

Furthermore, if an image being displayed in the layout area 42A is movedto the candidate area 41A (“YES” at step 56), as indicated by arrow 47Cin FIG. 12, it is judged that the characteristics of this image were notpreferable after all and therefore the weighting coefficients, which hadbeen raised, are restored to their original values (step 57). Further,if an image being displayed in the deletion area 43A is moved to thecandidate area 41A (“YES” at step 56), as indicated by arrow 47D in FIG.12, it is judged that the characteristics of this image were notdisliked after all and therefore the weighting coefficients, which hadbeen lowered, are restored to their original values (step 57).

When weighting coefficients are changed by page editing in the mannerdescribed above, the images in all of the image lists are scored againutilizing the weighting coefficients that have been changed (step 29 inFIG. 3). The weighting coefficients for an image having featurequantities in line with user preference are changed so as to raise thescore of the image, and the weighting coefficients for an image havingfeature quantities disliked by the user are changed so as to lower thescore of the image. Accordingly, since the scores of images in line withuser preference are raised, images in line with user preference are laidout in the virtual page 44 of virtual layout area 42.

When a page number n is designated, a page image in which images in linewith user preference have been laid out is actually displayed on thedisplay screen, as mentioned above. In this way files representing theelectronic album (photobook) are generated and the electronic album iscreated.

FIGS. 13 to 18 illustrate another embodiment and relate to a photobookcreation system constituted by a photobook creation apparatus 60 and animage search server 61 that communicate over a network.

The photobook creation apparatus 60 and image search server 61 arecapable of communicating with each other via a network such as theInternet.

The photobook creation apparatus 60 and image search server 61 have aconfiguration the same as that shown in FIG. 1. Naturally, the imagesearch server 61 would not require such components as thedisplay-related devices and memory card reader 8.

FIGS. 14 to 17 are flowcharts illustrating processing executed by thephotobook creation apparatus 60 and image search server 61.

Although all of the above-described processing is executed in thephotobook creation apparatus 1, the processing described below is suchthat page editing is performed in the photobook creation apparatus 1 andlayout in the image search server 61.

In a manner similar to that described above, the period of image captureis set in the photobook creation apparatus 60 (step 71 in FIG. 14) andimage files are read from the memory card 9 loaded in the photobookcreation apparatus 60 (step 72 in FIG. 14). The read image files and thedata representing the set period of time are sent from the photobookcreation apparatus 60 to the image search server 61 (step 73 in FIG.14). It may be so arranged that only image files representing imageswithin the set time period are sent from the photobook creationapparatus 60 to the image search server 61.

When image files representing a large number of images and dataindicating the set time period are received by the image search server61 (step 81 in FIG. 14), these images are grouped and an image list iscreated for each group (step 82 in FIG. 14) in a manner similar to thatdescribed above. The values of the feature quantities of all images inall of the lists are calculated (step 83 in FIG. 14) and a photobookpage number n is set to 1 (step 84 in FIG. 14). The feature quantityweighting coefficients are initialized (step 85) and all pages are addedto a list of unviewed pages (step 86 in FIG. 14).

An nth page (first page) is deleted from the unviewed page list (step 87in FIG. 15) and the images in all image lists are scored utilizing thefeature quantity weighting coefficients (step 88). In a manner similarto that described above, images having the top ten high scores in thetwo-page spread of the unviewed page list are laid out in the virtuallayout area 42, and images whose scores rank 11^(th) and lower areplaced in the virtual candidate area 41 (steps 89, 90 in FIG. 15).

An HTML (HyperText Markup Language) file representing the editing image(see FIG. 12) of the nth page also is generated and the HTML file issent from the photobook creation apparatus 1 to the photobook creationapparatus 60 (step 91 in FIG. 15).

If, when the HTML file sent from the image search server 61 is receivedby the photobook creation apparatus 60, the editing of the nth page hasnot been completed (“NO” at step 74 in FIG. 17), page editing is carriedout (step 75 in FIG. 17). The details of page editing will be describedlater. If page editing has not been completed (“NO” at step 76 in FIG.17), a page number is designated (step 78 in FIG. 17) and datarepresenting the designated page number is sent to the image searchserver 61 (step 79 in FIG. 17). When editing of all pages is finished(“YES” at step 76 in FIG. 17), a completion command is sent to the imagesearch server 61 (step 77 in FIG. 17).

FIG. 18 is an example of a flowchart illustrating page editingprocessing (the processing executed at step 75 in FIG. 17). Processingsteps in FIG. 18 identical with those shown in FIG. 11 are designated bylike step numbers.

With reference to FIG. 18, when the HTML file of the nth page sent fromthe image search server 61 is received by the photobook creationapparatus 60, the editing image 40A is displayed on the display screenof the photobook creation apparatus 60 (step 51 in FIG. 18), asillustrated in FIG. 12.

When an image is moved from the candidate area or deletion area to thelayout area (“YES” at step 52), in a manner similar to that set forthabove, a command that changes the weighting coefficients of the featurequantities that correspond to the feature quantities of this image issent to the image search server 61 (step 53A). When an image is movedfrom the candidate area or layout area to the deletion area (“YES” atstep 54), a command that lowers the weighting coefficients of thefeature quantities that correspond to the feature quantities of thismoved image is sent to the image search server 61 (step 55A). When animage is moved from the deletion area or layout area to the candidatearea (“YES” at step 56), a command that restores the changed weightingcoefficients to the original weighting coefficients is sent to the imagesearch server 61 (step 57A).

With reference to FIG. 16, when a command to change weightingcoefficients is received by the image search server 61 (“YES” at step92), the weighting coefficients are changed in accordance with thiscommand (step 93). If a completion command is not received (“NO” at step94), the image search server 61 checks to determine whether or not datarepresenting page number n has been received (step 95). When datarepresenting page number n is received (“YES” at step 95), page number nis set to a designated page number (step 96). Subsequent processing issimilar to that described above.

In the foregoing embodiment, it may be arranged so that a trash can(area) for unwanted persons is provided on the display screen. If animage containing a person from among images being displayed on thedisplay screen is moved to such a trash can for unwanted persons, theresult will be a layout in which this person is not included in thephotobook. In order to achieve this, the feature quantities indicativeof the likenesses of individual persons are detected for every image,and the feature quantities indicative of the likenesses of theseindividuals (the feature quantities of Mr. A, the feature quantities ofMr. B and the feature quantities of Mr. C and so forth) are stored inthe feature quantity table shown in FIG. 6. A person contained in animage that has been moved to the trash can for unwanted persons isspecified. By enlarging, in the negative direction, the weightingcoefficients applied to the feature quantities of this specified person,the image of the person contained in the image that has been moved tothe trash can for unwanted persons will not be laid out.

Of course, a trash can for unwanted persons need not be provided anew.If the feature quantities of every person are entered beforehand in thefeature quantity table shown in FIG. 6, then, by moving an imagecontaining an unwanted person to the above-described deletion area,there will be no rise in the score of the image containing this unwantedperson. As a result, the image containing the unwanted person can beprevented from being laid out.

FIGS. 19 to 23 illustrate a further embodiment, which is for the purposeof finding the image of a desired commercial product. In the systemconfiguration shown in FIG. 13, a client computer is utilized instead ofthe photobook creation apparatus 60. The configuration of the clientcomputer is the same as that shown in FIG. 1.

FIGS. 19 to 22 are flowcharts illustrating processing executed by theclient computer and image search server 61.

The client computer is connected to the Internet and accesses a desiredshopping site. If a first keyword (“clock”, for instance) is input tothe client computer (step 101 in FIG. 19), data representing the enteredfirst keyword is sent to the image search server 61 (a server for theshopping site) (step 102).

When the data representing the first keyword is received by the imagesearch server 61, images corresponding to the first keyword are readfrom an image database and a first image list is created (step 121 inFIG. 19). The values of feature quantities of all images in the createdfirst image list are calculated in the manner described above (see FIG.6). Further, the feature quantity weighting coefficients are initialized(step 123 in FIG. 19). The images in the image list are scored utilizingthe feature quantity weighting coefficients (step 124 in FIG. 19).

An HTML file representing an HTML page in which images having the topten scores will be laid out in a search result display area and imageswhose scores rank 11^(th) and lower will be placed in a candidate areais created (step 125 in FIG. 21). The HTML file created is sent from theimage search server 61 to the client computer (step 126 in FIG. 21).

When the HTML file sent from the image search server 61 is received bythe client computer, the search result image represented by this HTMLfile is displayed on the display screen of the display device of theclient computer (step 103 in FIG. 19). Next, a second keyword (“analogwristwatch”, for instance) is input to the client computer and datarepresenting the entered second keyword is sent from the client computerto the image search server 61 (step 105 in FIG. 19).

When the data representing the second keyword sent from the clientcomputer is received by the image search server 61, images that matchthe second keyword are selected from within the first image list and asecond image list is created (step 127 in FIG. 21). An HTML file inwhich images having the top ten scores from among the created secondimage list will be laid out in the search result area and images whosescores rank 11^(th) and lower will be placed in the candidate area iscreated (step 128 in FIG. 21). The HTML file created is sent from theimage search server 61 to the client computer (step 129 in FIG. 21).

When the HTML file sent from the image search server 61 is received bythe client computer, the search result image represented by this HTMLfile is displayed on the display screen of the display device of theclient computer (step 106 in FIG. 20).

FIG. 23 is an example of a search result image 140.

The search result image 140 includes a candidate area 141, a searchresult area 142 and a deletion area 143. Commercial product images 152having the top ten scores are displayed in the search result area 142,as mentioned above. Commercial product images 151 whose scores rank11^(th) and lower are displayed in the candidate area 141. If the searchis not finished (“NO” at step 107 in FIG. 20), then it is determinedwhether an image 150 being displayed in the search result area 142 hasbeen moved to the candidate area 141 (step 108 in FIG. 20) and, if not,whether the image 150 being displayed in the search result area 142 hasbeen moved to the deletion area 143 (step 109 in FIG. 20). If eithermovement is determined to have occurred, then a move commandrepresenting the content of this movement is sent from the clientcomputer to the image search server 61 (step 110 in FIG. 20). If an HTMLfile representing the search result image sent from the image searchserver 61 is received by the client computer, the search result image isdisplayed again; otherwise, control returns to the processing of step107 (step 111 in FIG. 20).

When the move command sent from the client computer is received by theimage search server 61 (“YES” at step 130 in FIG. 22) the weightingcoefficients of the feature quantities are updated in response to themove command (step 131 in FIG. 22). For example, in a case where theimage 150 being displayed in the search result area 142 has been movedto the candidate area 141, it is construed that the moved image 150closely resembles the desired product. As a consequence, the weightingcoefficients corresponding to the feature quantities of the moved image150 rise in value. In a case where the image 150 being displayed in thesearch result area 142 has been moved to the deletion area 143, it isconstrued that the moved image 150 does not resemble the desiredproduct. As a consequence, the weighting coefficients corresponding tothe feature quantities of the moved image 150 decline in value.

In this way the weighting coefficients are updated and the images in thesecond image list are scored. An HTML file in which images having thetop ten scores will be laid out in the search result area 142 and imageswhose scores rank 11^(th) and lower will be displayed in the candidatearea 141 is created (step 132 in FIG. 22). The HTML file created is sentto the client computer (step 133 in FIG. 22) and the desired productimage can be found at the client computer. If this is not the end of theprocessing of this flowchart (“NO” at step 134 in FIG. 22), processingfrom step 130 onward is repeated.

As many apparently widely different embodiments of the present inventioncan be made without departing from the spirit and scope thereof, it isto be understood that the invention is not limited to the specificembodiments thereof except as defined in the appended claims.

What is claimed is:
 1. An image search apparatus comprising: at leastone hardware processor configured to control a display unit to display aplurality of images in a candidate area of a display screen, calculate,with regard to each image of a multiplicity of images, the values offeature quantities representing characteristics of the image, score theimage based upon the values of the calculated feature quantities,responsive to application of a first move command which moves an imagedisplayed in the candidate area to a search result area, raise the valueof feature quantities, which correspond to the feature quantities of theimage for which the first move command has been applied and score saidmultiplicity of images based upon the raised values of the featurequantities, and determine image assignment in such a manner that apredetermined number of images having high scores are displayed in thesearch result area of the display screen, and images other than thepredetermined number of images are displayed in the candidate area ofthe display screen.
 2. The apparatus according to claim 1, wherein saidat least one hardware processor, in response to application of an imagedisplay command to display images in the determined assignment, controlsthe display unit so as to display the images in the decided placement.3. The apparatus according to claim 1, wherein said at least onehardware processor controls the display unit so as to display a deletionarea, in addition to the candidate area, on the display screen, andresponsive to application of a second move command which moves an imagebeing displayed in the candidate area or an image being displayed in thesearch result area to the deletion area, lowers the value of featurequantities, which correspond to the feature quantities of the image forwhich the second move command has been applied, and scores saidmultiplicity of images with the exception of said plurality of imagesbased upon the lowered values of feature quantities.
 4. The apparatusaccording to claim 1, wherein said at least one hardware processor, inresponse to application of a third move command which moves an image,which is being displayed in the search result area in accordance withthe first move command, to the candidate area, scores the images uponrestoring raised values of the feature quantities to original values. 5.The apparatus according to claim 1, wherein said at least one hardwareprocessor, in response to application of a third move command whichmoves an image, which is being displayed in the search result area inaccordance with the second move command, to the candidate area, scoresthe images upon restoring lowered values of the feature quantities tooriginal values.
 6. The apparatus according to claim 1, wherein the atleast one hardware processor creates an electronic album using imagesdisplayed in the search result area.
 7. An image search serverconstituting an image search system having a client computer and theimage search server, comprising: a feature quantity calculation devicefor calculating, with regard to each image of a multiplicity of images,the values of feature quantities representing characteristics of theimage; a scoring device for scoring the image based upon the values ofthe feature quantities calculated by said feature quantity calculationdevice; wherein a plurality of images having high scores obtained bysaid scoring device are displayed in a search result area of a displayscreen of said client computer and images other than the plurality ofimages are displayed in a candidate area of the display screen; and afirst scoring control device, responsive to application of a first movecommand which moves an image being displayed in the candidate area,which has been formed on the display screen of said client computer, tothe search result area, for controlling said scoring device so as toraise the value of feature quantities, which correspond to the featurequantities of the image for which the first move command has beenapplied, and score said multiplicity of images based upon the raisedvalues of the feature quantities.
 8. The server according to claim 7,further comprising a second scoring control device, responsive toapplication of a second move command which moves an image beingdisplayed in the candidate area, which has been formed on the displayscreen of said client computer, or an image being displayed in thesearch result area, to a deletion area, for controlling said scoringdevice so as to lower the value of feature quantities, which correspondto the feature quantities of the image for which the second move commandhas been applied, and score said multiplicity of images based upon thelowered values of feature quantities.
 9. A method of controllingoperation of an image search apparatus, comprising steps of: controllinga display unit so as to display a plurality of images in a candidatearea of a display screen; calculating, with regard to each image of amultiplicity of images, the values of feature quantities representingcharacteristics of the image; scoring the image based upon thecalculated values of the feature quantities; in response to applicationof a first move command which moves an image being displayed in thecandidate area to a search result area, raising the value of featurequantities, which correspond to the feature quantities of the image forwhich the first move command has been applied, and scoring saidmultiplicity of images based upon the raised values of the featurequantities; and deciding image placement in such a manner that apredetermined number of images having high scores obtained by scoringare displayed in the search result area of the display screen, andimages other than the predetermined number of images are displayed inthe candidate area of the display screen.
 10. A method of controllingoperation of an image search server constituting an image search systemhaving a client computer and the image search server, comprising stepsof: calculating, with regard to each image of a multiplicity of images,the values of feature quantities representing characteristics of theimage; scoring the image based upon the calculated values of the featurequantities; wherein a plurality of images having high scores obtained byscoring are displayed in a search result area of a display screen of theclient computer and images other than the plurality of images aredisplayed in a candidate area of the display screen; and in response toapplication of a first move command which moves an image being displayedin the candidate area, which has been formed on the display screen ofthe client computer, to the search result area, raising the value offeature quantities, which correspond to the feature quantities of theimage for which the first move command has been applied, and scoringsaid multiplicity of images based upon the raised values of the featurequantities.
 11. A non-transitory computer readable medium storing acomputer-readable program for controlling a computer of an image searchapparatus so as to: control a display unit so as to display a pluralityof images in a candidate area of a display screen; calculate, withregard to each image of a multiplicity of images, the values of featurequantities representing characteristics of the image; score the imagebased upon the values of the feature quantities calculated by saidfeature quantity calculation device; in response to application of afirst move command which moves an image being displayed in the candidatearea to a search result area, raise the value of feature quantities,which correspond to the feature quantities of the image for which thefirst move command has been applied, with regard to said multiplicity ofimages with the exception of said plurality of images, and score saidmultiplicity of images based upon the raised values of the featurequantities; and decide image placement in such a manner that apredetermined number of images having high scores obtained by scoringare displayed in the search result area of the display screen, andimages other than the predetermined number of images are displayed inthe candidate area of the display screen.
 12. An image search serverconstituting an image search system having a client computer and theimage search server, comprising: at least one hardware processorconfigured to calculate, with regard to each image of a multiplicity ofimages, the values of feature quantities representing characteristics ofthe image; score the image based upon the values of the featurequantities calculated by said feature quantity calculation device, aplurality of images having high scores being displayed in a searchresult area of a display screen of said client computer and images otherthan the plurality of images being displayed in a candidate area of thedisplay screen; and responsive to application of a first move commandwhich moves an image being displayed in the candidate area, which hasbeen formed on the display screen of said client computer, to the searchresult area, raise the value of feature quantities, which correspond tothe feature quantities of the image for which the first move command hasbeen applied, and score said multiplicity of images based upon theraised values of the feature quantities.